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Abstract 

Science teacher educators in the United States are currently preparing future science teachers to 
effectively implement the Next Generation Science Standards (NGSS) and, in thirteen states, to 
successfully pass a content-specific high stakes teacher performance assessment, the edTPA. 
Science education and teacher performance assessment experts have developed a crosswalk 
designed to highlight specific tasks within the secondary science edTPA; in these tasks, pre-service 
science teachers are prompted to plan, teach and assess their students as they engage in learning 
science explicitly aligned with the NGSS. The researchers in this study used qualitative methods 
to analyze archived actual pre-service science teacher edTPA portfolio artifacts to test the efficacy 
of this crosswalk. Evidence of student engagement in the NGSS scientific practices was found, 
confirming some components, and modifications of the NGSS and edTPA Crosswalk are 
suggested based on the results. 
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Introduction 

Ensuring that future science teachers have the knowledge and skills to implement the Next 
Generation Science Standards (NGSS) (NGSS Lead States, 2013a) is one of many challenges 
facing science teacher educators today. “Achieving the goals of the NGSS will take a long-term 
systematic effort that requires significant changes in instruction, curriculum, assessment, teacher 
preparation and professional development, accompanied by extensive financial, administrative, 
and public support” (National Science Teachers Association, 2013, p. 3). In addition to the charge 
from the National Science Teachers Association (NSTA), educator preparation programs are 
increasingly being challenged by legislators, external entities, and accreditors to meet higher 
requirements and to increase accountability (CAEP Board of Directors, 2013; Crowe, 2011; 
Darling-Hammond, 2010; Greenberg, McKee, & Walsh, 2013; US Department of Education, 
2009). High quality, nationally available assessments that reflect expectations for the job and meet 
rigorous standards for reliability and validity are just one component of the variegated pattern of 
accountability. Alignment with standards is necessary for both effective assessment elements 
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(Webb, Herman, & Webb, 2007) and effective teacher evaluation systems (Heneman & 
Milanowski, 2003). In January 2015, a crosswalk with a supporting evidence document was 
developed to display the intersection between a nationally available teacher performance 
assessment, the edTPA, and the science and engineering practices in the NGSS. (Brownstein, 
Stansbury, Whittaker, & Horvath, 2015). The initial NGSS and edTPA Crosswalk describes the 
extent to which edTPA explicitly prompts or provides opportunities for candidates to demonstrate 
teaching that reveals, supports, or develops NGSS practices. 

We asked the following question: does edTPA commentary provide evidence of NGSS 
scientific and engineering practices? For our inquiry, we analyzed edTPA planning, instruction, 
and assessment practice commentaries written by preservice science teachers for evidence of 
NGSS scientific and engineering practices. These commentaries are candidate reflections in 
response to edTPA prompts. An evidence-based understanding of the content linkages between 
NGSS scientific and engineering practices and edTPA commentaries may help prospective 
teachers prepare to implement the NGSS in K-12 settings. If science teacher educators understand 
which NGSS science and engineering practices are evidenced in the written commentaries, this 
understanding can be used to inform preservice science teacher curricular decisions. For example, 
the crosswalk may indicate specific NGSS topical areas that need to be addressed at another point 
in the preparation program or during a new teacher’s induction years. For clarity, we will provide 
a brief explanation of the science and engineering practices in the NGSS (NGSS Lead States, 
2013b), edTPA’s design, and the initial NGSS and edTPA Crosswalk (Brownstein et al., 2015). 
Subsequent sections of the article describe the qualitative, constant comparative methodology; the 
results of the commentary content linkage analysis; a discussion of the interpreted results in 
relation to existing literature; and the implications for science teacher educators and the teacher 
education profession. 


Literature Review 

The Next Generation Science Standards (NGSS) 

The NGSS (NGSS Lead States, 2013a) are the current K-12 national performance-based 
science standards recommended for adoption by individual states. Developed with input from 
states and various stakeholders, including scientists, science educators, and education researchers, 
these standards build on the foundational work of the Framework for K-12 Science Education for 
the teaching and learning of science in the United States (NGSS Lead States, 2013a). Under the 
NGSS, students are expected to develop an integrated understanding of science that incorporates 
knowledge of specific scientific phenomena and the use of science and engineering practices to 
develop this knowledge (Carpenter, Iveland, Moon, & Bianchini, 2015). Students are also required 
to make connections, strengthening their knowledge through the use of crosscutting concepts such 
as cause and effect, structure and function, and energy and matter (Duschl, 2012). To fully realize 
the promise of the NGSS, a fundamental shift in how students engage in learning science is 
required (Duschl, 2012; National Research Council, 2011; Penuel, Harris, & DeBarger, 2015; 
Roseman, Fortus, Krajcik, & Reiser, 2015). 

According to the National Science Teachers Association (2013), 

One of the most significant shifts of the NGSS is the recommendation that students engage 
in science learning at the nexus of three dimensions: science and engineering practices, 
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crosscutting concepts, and disciplinary core ideas. Because many current state and district 
standards address these dimensions separately, it will take a considerable effort to embrace 
this new vision in the implementation of the NGSS, including instruction, curriculum, 
assessment, and teacher preparation and professional development (p. 2). 

Additionally, the NGSS were developed to demonstrate proficiency in science by establishing 
performance expectations and assessment recommendations (NGSS Lead States, 2013c). In 
essence, the NGSS challenges science educators to design and implement curricula with attention 
to “how science and engineering is practiced in the real world” (NGSS Lead States, 2013c, p. 1). 
It is imperative that science teacher educators develop mechanisms to support both pre-service and 
in-service teachers in engaging K-12 students in the science and engineering practices as a required 
component of the NGSS (National Research Council, 2011; Penuel et al., 2015; Wilson, 2013). 

The NGSS identifies eight science and engineering practices, which we refer to collectively 
as practices. The NGSS practices (NGSS Lead States, 2013b) are as follows: 

1. Asking questions and identifying problems 

2. Developing and using models 

3. Planning and carrying out investigations 

4. Analyzing and interpreting data 

5. Using mathematics and computational thinking 

6. Constructing explanations and designing solutions 

7. Engaging in argument from evidence 

8. Obtaining, evaluating, and communicating information 

In NGSS Appendix F, each practice includes specific grade band competencies. In the example 
below, analyzing and interpreting data is identified in 9- 12th grade classrooms through evidence 
of the following competencies: 

• “Analyzing data in 9-12 builds on K-8 experiences and progresses to introducing more 
detailed statistical analysis, the comparison of data sets for consistency, and the use of 
models to generate and analyze data. Analyze data using tools, technologies, and/or models 
(e.g., computational, mathematical) in order to make valid and reliable scientific claims or 
determine an optimal design solution. 

• Apply concepts of statistics and probability (including determining function fits to data, 
slope, intercept, and correlation coefficient for linear fits) to scientific and engineering 
questions and problems, using digital tools when feasible. 

• Consider limitations of data analysis (e.g., measurement error, sample selection) when 
analyzing and interpreting data. Compare and contrast various types of data sets (e.g., self- 
generated, archival) to examine consistency of measurements and observations. 

• Evaluate the impact of new data on a working explanation and/or model of a proposed 
process or system. 

• Analyze data to identify design features or characteristics of the components of a proposed 
process or system to optimize it relative to criteria for success” (NGSS Lead States, 2013b 
p.9) 

As stated in Appendix F of the Next Generation Science Standards, “The eight practices 
are not separate; they intentionally overlap and interconnect. . . Just as it is important for students 
to carry out each of the individual practices, it is important for them to see the connections among 
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the eight practices” (NGSS Lead States, 2013b, p. 3) 1 . Recognizing the overlap and the 
interconnectedness of the science and engineering practices within effective science teaching, the 
developers of the science edTPA handbooks (with input from one of the authors) strategically 
chose to incorporate multiple practices into the requirement for completing and passing this 
nationally available teacher performance assessment (Stanford Center for Assessment, 2014). 

edTPA 

Developed by experienced educators and the Stanford Center for Assessment, Learning 
and Equity (SCALE), edTPA is a nationally recognized high stakes performance assessment 
designed to assess preservice teachers’ readiness to teach; it has been evaluated to be valid and 
reliable (Pecheone, Shear, Whittaker, & Darling-Hammond, 2013; Sato, 2014; Stillman, Ragusa, 
& Whittaker, 2015). Through its widespread adoption, edTPA supports the professionalization of 
teaching by creating a recognized standard for clinical preparation that mirrors standards for other 
professional fields such as engineering, medicine, or accounting (Darling-Hammond, 2010). 
edTPA assesses developing pedagogical content knowledge (PCK) with explicit attention to 
student context, and consists of three interrelated tasks: planning, instruction, and assessment 
(Pecheone et al., 2013; Stillman et al., 2015). Completed during student teaching, edTPA requires 
the teaching-credential candidate to produce a portfolio with evidence-based reflective 
commentaries focused on three to five consecutive hours of teaching. To complete a secondary 
science edTPA, preservice science teachers must, from prompts, produce a set of reflective 
commentaries that describe their instructional context as they address, analyze and justify three 
specific aspects of their teaching: planning of the learning segment, engaging students in learning, 
and assessing student learning. The required supporting evidence is composed of video clips of 
classroom interactions during the learning segment, student work samples from the assessment 
analysis, instructional materials, and lesson plans. Once submitted, an edTPA portfolio is scored 
across 15 rubrics by a trained and calibrated scorer who has validated teaching experience and 
expertise in the grade level and content area of the subject edTPA (Adkins, Spesia, & Snakenborg, 
2015; Pecheone et al., 2013). 

The secondary and middle school science edTPA handbooks were written by SCALE with 
input from science educators and assessment experts (Stanford Center for Assessment, 2014). The 
edTPA secondary school science design team intentionally integrated some NGSS science and 
engineering practices into the rubrics and prompts to support pre-service teacher candidates in 
engaging their students in learning science with specific attention to the NGSS (Pecheone et al., 
2013; Stillman et al., 2015). The crosscutting concepts and disciplinary core ideas may also be 
evidenced in edTPA portfolios as components of the school/classroom curriculum. By necessity, 
the implementation of the cross-cutting concepts and disciplinary core ideas will vary widely from 
one teaching context to the next (by grade level and course). 

Next Generation Science Standards (NGSS) and edTPA Crosswalk 

“For a system to work ... its elements must be aligned. . . (I)f classroom teaching and 
learning activities are to help students attain the standards, they too must be aligned with the 
standards” (Herman & Webb, 2007, p. 3). Therefore, understanding how a nationally implemented 
preservice teacher assessment (edTPA) is linked to the NGSS may help guide science educators 


1 For a complete description of the eight science and engineering practices, see NGSS Appendix F (NGSS 
Lead States, 2013b). 
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in their preservice preparation programs. A crosswalk differs from an alignment chart. Alignment 
indicates a straight line between the assessment and the standards (Baker, 2004). In this case, 
edTPA is designed to evaluate authentic teaching practices and not NGSS practices. Therefore, 
true alignment is not an attainable or an appropriate goal. Instead, a crosswalk can be used to 
describe the intersections between the assessment and the standards. These linkages may be used 
to inform science educators of how edTPA can support the integration of NGSS practices within 
a K-12 experience. 

The NGSS and edTPA Crosswalk (Brownstein et al., 2015) was developed using an expert 
review methodology and inter-rater agreement (Webb et al., 2007). Two science teacher educators 
and two assessment and evaluation experts engaged in a content validation process to judge the 
extent to which the rubrics and/or prompts provide opportunities for candidates to demonstrate the 
NGSS practices (Beck, 2007; Martone & Sireci, 2009). These potential edTPA/NGSS content 
linkages were found in five of the eight scientific practices (see Appendix A). A complete version 
of the crosswalk includes the edTPA rubrics and/or the prompt language with potential content 
linkage to the NGSS practice (see https : //secure . aacte .or g) . 

Methodology 


Sample 

In this research, content linkage requires more than an analysis of the assessment against 
the standards. It is not only evidenced through an analysis of the edTPA prompts and rubrics but 
also cross-checked with actual preservice teachers’ edTPA commentary to provide a detailed and 
accurate picture of the intersections with the NGSS science and engineering practices. We 
analyzed the commentaries from nationally scored secondary science edTPA portfolios for all 
preservice science teacher candidates who completed a single subject credential program at a large 
western public urban university in the spring of 2014. The ten credential candidates, six women 
and four men, ranged in age from to 25 to 48, and all possessed a bachelor’s degree in a science 
content area prior to beginning the program. Three of the ten candidates entered the program 
immediately after completing their baccalaureate degree. Year-long clinical placements were in a 
large urban area at nine high schools and one middle school. Candidates were placed in an urban 
setting for 90 hours of clinical experience in the fall term and student teaching in the spring 
semester. Although all schools were urban, there were wide variations in their demographics, 
including small to large student populations, low to moderate socioeconomic status, diverse social 
cultures, and academic climate. By the time they submitted their edTPA portfolios, the preservice 
science teachers had completed ten weeks of the student teaching experience in the same classroom 
where they completed their initial clinical experience in the fall term. The submitted portfolios 
included biology, chemistry, and physics (see Table 1). 

Candidates had also completed one and a half semesters of required credential coursework 
while being enrolled in a two semester credential program. For spring 2014, the state mandated 
that each credential candidate must pass the edTPA with a score of 41 as a requirement for 
successful program completion and the award of a valid preliminary single subject science 
credential (California Commission on Teacher Credentialing, 2014). 
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Table 1 Science disc, for each portfolio 


edTPA Portfolio Portfolio Discipline Grade Level 

Port. #1 

Physics 

High School 

Port. #2 

Biology 

Middle School 

Port. #3 

Biology 

High School 

Port. #4 

Chemistry 

High School 

Port. #5 

Biology 

High School 

Port. #6 

Chemistry 

High School 

Port. #7 

Biology 

High School 

Port. #8 

Biology 

High School 

Port. #9 

Biology 

High School 

Port. #10 

Biology 

High School 


Data Collection 

All edTPA portfolios were submitted to be scored, and then archived, through the 
electronic platform, Taskstream. The researchers obtained the required permissions and the 10 
edTPA portfolios were released by the program’s edTPA coordinator. 

After data analysis was complete, scores and summarized demographic data were available 
to the researchers. Results ranged from 39 to 52 (out of 75, based on 15 rubrics, each with a five- 
point scale), where each score was reported as a total across all fifteen rubrics. The campus from 
which the portfolios were drawn was in the process of transitioning from the Performance 
Assessment for California Teachers (PACT) (Pecheone & Chung, 2006) to edTPA. 

The portfolio submitted by candidates includes responses to prompts in each of the three 
tasks (planning, instruction, and assessment) as well as supporting evidence of lesson plans, videos 
of lessons, and example student work (Pecheone et al., 2013). For this research, we analyzed 
candidate commentary from the three tasks. Because candidates submit lesson plans in a format of 
their own choosing, the lesson plans may not necessarily speak to what is understood by the 
prospective teacher nor consistently reflect what occurs in the classroom (Beck, 2007). 
Additionally, the official scorers use submitted lesson plans and videos as supporting evidence for 
candidate commentaries. Therefore, we chose not to code submitted lesson plans. Candidates 
document their teaching during the learning segment by submitting up to two video clips that span 
no more than 20 minutes. The submitted clips are selected by candidates to use as evidence of their 
ability to analyze, through prompted written commentaries, how they have engaged their students 
in learning. When the edTPA is scored for consequential purposes, video recordings provide 
supporting evidence for how the candidate engages with K-12 students. The submitted examples 
of student work are selected by the candidate to demonstrate the range of student work and are 
used to support their reflective commentaries. In this study, we focused on the candidate’s ability 
to apply NGSS practices as evidenced in his or her writing about and reflecting on his/her teaching. 
Therefore, in choosing to use candidate commentaries to guide our inquiry, we acknowledge that 
the identification of evidence for any of the eight practices is limited to what candidates write 
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about or analyze in their teaching rather than what they might actually enact as identified by an 
objective expert observer. 

Coding Scheme and Reliability 

Our study followed the protocols for qualitative research outlined in Merriam (2009) and 
Denzin and Lincoln (201 1). Inductive qualitative content methods were used for all data analyses 
(Denzin & Lincoln, 2011; Merriam, 2009). The categories for coding were drawn from the 
descriptions for each of the eight scientific and engineering practices found in Appendix F of the 
NGSS (2013b). For each category (science and engineering practice), the grade 6-8 competencies 
were used for the middle school portfolio, and the grade 9-12 competencies were used for the high 
school portfolios. For the remainder of the paper, we will refer to each category as one of the eight 
science and engineering practices. For each specific science and engineering practice, narrative 
passages from the candidates’ reflective commentaries were coded. 

Capturing both manifest and latent meaning of the content (Cho & Lee, 2014), coding 
indicated if the preservice teacher planned to engage students in that practice, actually did engage 
students in that practice, or reflected on missed opportunities to support student engagement in 
that practice. We utilized a qualitative content analysis by applying a deductive version of the 
constant comparative method to obtain accurate evidence in all eight coded practices (Cho & Lee, 
2014; Fram, 2013). The outcomes were intact text with indexing themes (Glaser & Laudel, 2013). 
Data analysis is limited to the NGSS practices categories and understanding themes, or links, from 
those categories (Graneheim & Lundman, 2004). 

For coding of the commentaries to the practices, Glaser and Strauss’ constant comparison 
methodology, comparing and contrasting consistent coding instances within the data (1967) while 
constantly checking for inter-coder agreement between the researchers (Schilling, 2006). 
Descriptions from the practices matrix in NGSS Appendix F were used to determine if the 
commentary reflected the quality of that particular practice. For a passage to be coded as evidence 
of a practice, the commentary needed to address the breadth, depth, and challenge of one bullet 
point within the grade band of the practice matrix. It was not sufficient to reference a practice 
without an elaboration of the application of said practice and an integration of the learning of 
science. Examples of commentary-edTPA coded content linkages are included in Table 2. In each 
example, the students or prospective teachers are actively engaged (either in the classroom or 
through writing) in an NGSS practice. In a coded portfolio, the student teacher may be working to 
make sense of the practice and to understand how it plays out in the classroom. For example, 
portfolio #8 in Table 2 has a passage coded as analyzing and interpreting data. The central concept 
is, “How are new species formed?” (portfolio #8, planning commentary). During this portion of 
the lesson, students are using the science practice to learn the science. In contrast, in portfolio #1, 
students keep a log of their work building a musical instrument and “use physics vocabulary from 
the waves unit to explain the scientific phenomena behind music and building a musical 
instrument” (portfolio #1, planning commentary). However, in the commentary, there was little 
evidence of physics in the logs. Therefore, while there were components of the lesson where the 
learning of science was integrated with a science practice, this particular component, the log, was 
not coded as an example of a P-12 student engaging in a practice. 
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Table 2 Example commentary for NGSS practices 

Commentary Example #1 


NGSS 

4 


NGSS 

7 


thumb) but didn’t use any evidence to back up 
his 

claim that this clan was most successful. 


Words from NGSS practices 
matrices 


Portfolio #8: Assessment Commentary 

As a class they did very well analyzing and 
organizing data to make a hypothetical 
phylogenetic tree. 84% of the class is at Level 4, 
meaning the tree is complete and incorporates 
the data in a logical way. All students attempted 
to make a tree. I think it really helped them that 
we practiced drawing trees several times with 
different data. They could see how the 
organization of the tree could change depending 
on what kind of data was used. Additionally, 
they saw me do an example on the board and 
some of their classmates’ work projected 
through the doc cam. I didn’t see much 
innovation with the trees, although I tried to 
impress upon them that evolutionary trees might 
look very different from each other and still say 
the same thing 

Portfolio #5: Assessment Commentary 

Students seemed comfortable picking a clan that 
was fittest- almost all were able to do this- but 
many students struggled with using evidence to 
back up their argument. . . . Student B only 
talked about what it waslike for the hand clan to 
collect beans (easier due to having an opposable 


Compare and contrast various 
types of data sets (e.g., self- 
generated, archival) to examine 
consistency of measurements and 
observations (NGSS Lead States, 
2013b, p. 9). 


Construct, use, and/or present an 
oral and written argument or 
counter-arguments based on data 
and evidence (NGSS Lead States, 
2013b, p. 13). 


To ensure consistency of coding using the NGSS practices matrix, we examined all three 
tasks in the ten portfolios for each practice separately. Lor the four science and engineering 
practices with potential content linkages as shown in Appendix A, we validated our level of coding 
consistency using two portfolios until our coding matched. We then independently coded the 
remaining eight portfolios and discussed our rate of agreement for each code incident to ensure the 
validity of our codes and to confirm inter coder reliability (Boeije, 2002; Webb et al., 2007). Lor 
the remaining four science and engineering practices, we used the process described above to 
independently code two full portfolios as well as to check and discuss our rate of agreement for 
each practice. We continued to code a portfolio until we reached a level of agreement above 90% 
for all three tasks within a single portfolio. We then divided the remaining portfolios and 
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independently finished coding each of the four remaining practices. After we repeated our process 
for all eight practices, we chose random portfolios under each practice for a final reliability check 
and found 90% or greater agreement (Webb et al., 2007; Westbrook, 1994). At the completion of 
the process described above, each portfolio had been analyzed for coding a minimum of eight 
times and a maximum of 32 times. 


Results 

The 184 pages of portfolio commentaries provided rich data with which to identify 
evidence of specific science and engineering practices embedded in the teaching of science 
content. Table 3 presents the tabulation of the coded instances of each practice within each 
portfolio. The frequency of coded instances was classified into four levels: no linkages, minimal 
linkages, moderate linkages, and substantive linkages. Results with minimal linkages were re- 
examined to determine if a higher-level classification was warranted due to extensive breadth and 
depth. No portfolio levels were changed. 

Table 3 NGSS practice frequency within each portfolio 


edTPA 

Portfolio 

NGSS Practices 

1. Asking 
Qs& 
defining 
problems 

2. 

Developi 

ng ti- 
ming 
models 

3. Planning 
& carrying 
out 

investigation 

s 

4. 

Analyzing 
& interp. 
data 

5. Using 
math & 
comp, 
thinking 

6. 

Constructing 
explanations 
& designing 
solutions 

7. Engaging 
In argument, 
from 
evidence 

H. Obtaining, 
evaluating & 
zommun. info. 

Port. #1 

4 

4 

<-» 

t 

t 

t 

0 

t 

Port. #2 

0 

0 

t 

t 

4 

t 

t 

<-» 

Port. #3 

0 

t 

4 

t 

<-» 

t 

t 

t 

Port. #4 

4 

0 

4 

t 

t 

t 

<-» 

t 

Port. #5 

0 

t 

t 

t 

<-» 

<-» 

t 

t 

Port. #6 

0 

t 

t 

t 

t 

t 


t 

Port. #7 

0 

0 

4 

0 

0 

t 

4 

t 

Port. #8 

0 

t 

t 

t 

<-» 

t 

t 

t 

Port. #9 

0 

t 

0 

t 

0 

t 

t 

t 

Port. #10 

0 

t 

0 

t 

t 

t 

t 

t 


Portfolio Commentary Coding Definition Key 


Symbol 

Code 

Number of instances 

0 

None 

0 


Minimal 

1-3 


Moderate 

4-7 

t 

Substantive 

7 or more 


We then examined the levels of occurrence for each of the practices across the ten 
portfolios. If a practice was deemed to be substantive, it not only needed to have sufficient 
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representation of the practice but also the practice had to be demonstrated consistently across the 
ten portfolios. We found consistent and strong content linkages for three of the five practices 
originally identified as linked in the crosswalk (see Table 4). The remaining two identified 
practices were identified as having inconsistent content linkages. The three practices noted in the 
crosswalk as not being linked had weak or inconsistent evidence. 

Table 4 NGSS science and engineering practice level of content linkage to edTPA portfolio 
evidence 


NGSS Science and Engineering Practice Level of 

Linkage 

1. Asking questions and defining problems 

Weak 

2. Developing and using models 

Inconsistent 

3. Planning and carrying out investigations 

Inconsistent 

4. Analyzing and interpreting data 

Strong 

5. Using mathematics and computational thinking 

Inconsistent 

6. Constructing explanations and describing solutions 

Strong 

7. Engaging in argumentation from evidence 

Inconsistent 

8. Obtaining, evaluating and communicating information 

Strong 


Discussion 

To confirm content linkage, an assessment (edTPA) must consistently show evidence of a 
connection to the particular standard (i.e., science and engineering practice) (Porter, 2002). 
Although we are pleased to confirm strong content linkages between edTPA commentaries and 
three of the NGSS practices, we need to address the two practices originally identified as linked 
but with inconsistent content linkages demonstrated by the candidate commentary evidence: 
planning and carrying out investigations and engaging students in scientific argumentation. This 
does not mean that edTPA does not prompt candidates to plan for or to engage students in these 
practices but that candidates did not provide consistent evidence of actually doing so. 

One originally identified linked practice with inconsistent evidence was planning and 
carrying out investigations. This practice is well established as an integral component of best 
practices in science teaching (DeBoer, 2004; Dewey, 1910, 1916), although it is yet to be widely 
applied in science classrooms (Duschl & Bybee, 2014). The NGSS practice of planning and 
carrying out investigations states, “Students should design investigations that generate data to 
provide evidence to support claims they make about phenomena. Data aren’t evidence until used 
in the process of supporting a claim. Students should use reasoning and scientific ideas, principles, 
and theories to show why data can be considered evidence” (NGSS Lead States, 2013b, p. 7). 
Coding the commentaries required that they clearly demonstrated specific criteria in the NGSS 
practices matrices. One of the planning and carrying out investigations 9-12 criteria is as follows: 
“Plan and conduct an investigation individually and collaboratively to produce data to 
serve as the basis for evidence, and in the design: decide on types, how much, and accuracy 
of data needed to produce reliable measurements and consider limitations on the precision 
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of the data (e.g., number of trials, cost, risk, time), and refine the design accordingly” 

(NGSS Lead States, 2013b, p. 7). 

To be coded, the commentary would need to include each component. In the context of the 
edTPA guidelines, which require three to five hours of continuous instruction, it may be unrealistic 
to expect candidates that teach in a variety of middle and secondary content areas to devote 
significant, substantive attention to planning and carrying out full investigations in addition to the 
other edTPA requirements. While we recognize that participation in, and the effective 
implementation of, all eight practices is essential to fully address the NGSS (NGSS Lead States, 
2013a), the student teaching context and scope required by edTPA necessarily limits what can 
reasonably be required of a candidate in building a portfolio. However, we suggest that when 
viewed over the ten to fifteen weeks of student teaching typically experienced by preservice 
science teachers during their clinical work, it is possible to create opportunities for the candidates 
to practice and refine engagement with their students in planning and carrying out investigations 
outside of edTPA. 

A second practice identified in the original crosswalk with inconsistent evidence was 
engaging students in scientific argumentation. This practice is not common in classrooms and has 
been identified as a key/core practice that will require significant attention and reflective planning 
to successfully engage science students (Berland & Reiser, 2011; Driver, Newton, & Osborne, 
2000). Given specific attention to argumentation in prompts and rubrics in edTPA (see Appendix 
A), we expected to see evidence of argumentation. However, as reported in the results, we found 
the coded instances for the practice of engaging in argument from evidence to be an inconsistent 
pattern of content linkage, with most occurring in the biology portfolios; we also identified a gap 
in the planning commentaries. The cause behind this lack of candidate demonstration of this NGSS 
practice is unclear: do edTPA prompts need to be more explicit, do candidates not understand the 
practice itself, do candidates not engage in this practice during student teaching, or does only 
particular science content lends more to the scientific argumentation practice? To better ascertain 
the cause of this limitation, we recommend examining the planning commentary prompts as well 
as the planning rubric in the science edTPA handbooks with an eye to bolstering credential 
candidates’ attention to argumentation in planning, which could lead to more substantive attention 
in both instruction and assessment. Additionally, with changes in the prompts and rubrics, a future 
study could better determine the source of the inconsistent results. 

For practices not identified as linked in the crosswalk, results varied. For example, we 
found weak content linkage for the practice of asking questions and defining problems. This result 
is consistent with the original crosswalk and may be due to the high level of performance 
expectations within the NGSS practices matrix: “Asking questions and defining problems in 9-12 
. . . [is] formulating, refining, and evaluating empirically testable questions and design problems 
using models and simulations” (NGSS Lead States, 2013b, p. 4). 

However, we did find evidence of inconsistent content linkage for both the practice of 
developing and using models and the practice of using mathematics and computational thinking. 
For the practice of developing and using models, we recognize that model-based reasoning is a 
long standing core practice in science and in the teaching of science (DeBoer, 1991; Finlay, 1962; 
Halloun & Hestenes, 1984; Kuhn, 1970; Moore, 1968; National Research Council, 1996; 
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Passmore, Stewart, & Cartier, 2009; Rutherford & Ahlgren, 1990; Svoboda & Passmore, 2013), 
so we are not surprised to find evidence of the practice. However, the lack of specific prompts or 
rubric criteria may have contributed to the inconsistent evidence. For the inconsistent linkages in 
the practice of mathematics and computational thinking, there were a relatively high total number 
of coded instances (Miles & Huberman, 1994). Although evidence of the practice of using 
mathematics and computational thinking was found across eight of the ten portfolios, nearly two 
thirds of the identified instances of the practice were in the three physical science portfolios. 
Additionally, the literature describes the NGSS mathematical reasoning practice as emphasizing 
deep student thinking rather than algorithmic thinking (Mayes & Koballa Jr, 2012; Te kk umru- 
Kisa, Stein, & Schunn, 2015). While the content linkages were coded, it was not clear if the items 
represented engagement in the practice as deep student thinking or meaning making. Therefore, it 
is our observation that the content linkage of edTPA to the mathematics and computational 
thinking practice is unclear. 


Implications 

Our literature review, crosswalk including rubrics and prompts for opportunities to 
demonstrate NGSS practices, and research findings on content linkage between edTPA secondary 
science portfolios and the NGSS have multiple implications. While our study is limited to a single 
setting, these results provide examples of novice teachers’ understanding the application of NGSS 
practices. One indication for the educator preparation community is to consider the advantages of 
using crosswalks to empower professional educators to examine and utilize the content linkages 
between edTPA and standards as well as to inform educators in preservice teacher curricular 
decision making. According to the National Research Council, “the assessments required for 
teacher licensure and the course work needed for subject-area certification need to reflect the types 
of learning and assessment tasks that teachers will be expected to develop for students” (National 
Research Council, 2011, p. 83). In this vein, crosswalks may be useful to verify content linkages 
between edTPA and the Council for the Accreditation of Educator Preparation (CAEP) Specialty 
Professional Associations (SPA) content- specific standards for preparing educators, K-12 content 
standards, and state teaching standards. 

According to Herman and Webb (2007), “For the system to work, however, its elements 
must be aligned” (p. 3). Crosswalk development requires a rigorous protocol that includes 
validation of identified content linkages through analysis of candidate portfolios. In this paper, a 
protocol that integrated a qualitative deductive version of the constant comparative methodology 
(Cho & Lee, 2014; Fram, 2013) with assessment alignment procedures (Martone & Sireci, 2009) 
was developed and followed. Our methodology is unique in contrast with traditional methods for 
evaluating alignment between standards and assessment in that we analyzed for content linkages 
and not complete alignment (see Table 5). Based on this research, an updated crosswalk has been 
developed (see Appendix B) and posted on the AACTE edTPA secure website (Brownstein et al., 
2015). Additionally, an interesting question to pursue may be to examine the curriculum taught in 
the edTPA portfolio for disciplinary core ideas and crosscutting themes, but that is beyond the 
focus of this project. While these are essential components of NGSS, a study of the implementation 
of these two NGSS dimensions is better suited to an examination of the curriculum and 
observations in K-12 classrooms over an extended period of time. 
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Table 5 edTPA to NGSS practice content linkage: Proposed to evidence-based 


NGSS Science / Engineering Practice Originally Final evaluation 

identified of content linkage 
content based on 

linkage to candidate 

edTPA evidence 

1. Asking questions and defining problems 

No 

No 

2. Developing and using models 

No 

No 

3. Planning and carrying out investigations 

Yes 

No 

4. Analyzing and interpreting data 

Yes 

Yes 

5. Using mathematics and computational thinking 

No 

No 

6. Constructing explanations and describing solutions 

Yes 

Yes 

7. Engaging in argumentation from evidence 

No 

No* 

8. Obtaining, evaluating and communicating information 

Yes 

Yes 


^Recommended changing rubric and prompts to more clearly address NGSS practice 


The development and analyses of crosswalks between standards and assessments has the 
potential to support thoughtful, faculty inquiry (question-oriented) approaches to the 
implementation of edTPA in teacher preparation programs. Peck, Gallucci, and Sloan (2010) write 
about the potential pitfalls of a ‘compliance’ approach to implementing edTPA and the potential 
benefits of a faculty inquiry (question-oriented) approach. A compliance approach focuses on the 
outcomes and accountability mandates for candidate completion and passing of edTPA and can 
possibly narrow the program goals for both the teacher educators and the candidates (Peck et al., 
2010). Conley and Gardner (2015) build on their work and describe a faculty inquiry approach: 
“In this case, edTPA was positioned as a vehicle for generative thought among the teacher 
educators and the teacher candidates” (p. 5). As educators, we can use the knowledge of the 
intersections between edTPA and the NGSS practices to support and build the needed changes in 
the teaching of science through the use of a science education faculty driven inquiry approach. 

For science educator preparation programs, edTPA provides only partial evidence of 
science credential candidates’ implementation of the NGSS practices. If we wish to better prepare 
our future science teachers to engage K-12 students in the NGSS practices, we will need 
assessments that can provide evidence for the remaining four practices. An assessment that fills in 
the NGSS gaps in edTPA should be developed for use by educators throughout the US and 
territories. Peterson and Bruster do note that edTPA also strongly supports reflective practice 
(2014). There is a potential that preservice science teachers completing the science edTPA may 
not be just demonstrating an understanding of single specific NGSS science and engineering 
practices, but through reflective responses prompted by edTPA in the context of their actual 
practice, preservice science teachers may be increasing their understanding and ability to 
implement a suite of NGSS science and engineering practices. Still, if we wish to better prepare 
future science teachers to engage k-12 students in the full 3-dimensional nature of the NGSS, we 
will need more comprehensive PCK centered assessments that also focus on the cross cutting 
concepts and specific Disciplinary Core Ideas. Assessments that fill in the NGSS gaps in edTPA 
should be developed and shared by science educators both in the US and internationally. 
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Additionally, science teacher educators’ need to advocate for a disciplinary- specific faculty 
inquiry approach to the implementation of edTPA. Because all learning is a “process of active 
construction” (Cochran-Smith & Villegas, 2015, p. 10), it is possible for science teacher educators 
to support greater understanding of the NGSS practices through active formative preparation, 
enabling the summative creation of a strong, reflective science edTPA portfolio. We agree with 
Stroupe (2015) that there need to be “more studies to better understand . . . how to help novice 
educators use their science learning experiences to design similar opportunities for students.” (p. 
1038). The work in this paper contributes to the research needed to prepare candidates for the 
changes necessary to engage students in science as required by the NGSS (National Science 
Teachers Association, 2013). Capitalizing on opportunities to engage preservice teachers in critical 
reflective practice around the suite of edTPA linked practices, both in the program coursework as 
well as in clinical practice, becomes imperative for success on the edTPA and their readiness to 
engage students in learning science through inquiry-based practices that reflect how science and 
engineering are done in the real world (National Science Teachers Association, 2013). Refining 
the crosswalk to better fit the identified content linkages between the NGSS and edTPA, 
strengthening the prompts for argumentation in the science edTPA handbook, and taking a faculty 
inquiry approach to edTPA implementation will support the preparation of highly qualified science 
teachers who are ready to meet the challenges of implementing the NGSS. 
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Appendix A 

[Original] Next Generation Science Standards (NGSS) and edTPA Crosswalk 

This crosswalk is a representation of where the NGSS are demonstrated in the edTPA rubrics and prompts. See original for alignment analysis. 



edTPA Rubric 

NGSS Practices 

1 . Asking 

Qs & 

defining 

oroblems 

2. Developing 
& using 
models 

3. Planning & 
carrying out 
investigations 

4. Analyzing 
& 

interpreting 

data 

5. Using math 
& 

computational 

thinking 

6. Constructing 
explanations & 
descriptions 
solutions 

7. Engaging in 
argumentation 
from evidence 

8. Obtaining, 
sval., & 
communicating 
information 

Task 1: Planning 

I. Planning for Conten t 
Understandings 



V 



V 

V 

V 

2. Planning to Support Varied 

Student Needs 









3. Using Knowledge of Students to 
Inform Planning 









4. Identifying and Supporting 
Language Demands 









5. Planning Assessments to Monitor 
and Support Student Learning 









Task 2: 

6. Demonstrating a Positive and 
Engaging Learning Environment 









7. Engaging Students in Learning 




V 


V 

V 

V 

8. Deepening Student Learning While 
Teaching 






V 


V 

9. Subject Specific Pedagogy: Science 




V 





10. Analyzing Teaching 









Task 3: Assmt. 

11. Analyzing Student Work 









12. Providing Feedback to Guide 
Learning 









13. Supporting Students ’ Use of 
Feedback 









14. Evidence of Language Use to 
Support Content Understandings 






V 



15. Using Assessment to Inform 
Instruction 







V 



Retrieved from https://secure.aacte.org 
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Appendix B 

[SUGGESTED UPDATE] Next Generation Science Standards (NGSS) and edTPA Crosswalk 


This crosswalk is a representation of where the NGSS are demonstrated in the edTPA rubrics and prompts as well as evidenced in candidate planning, 
instruction, and assessment commentaries. See original document for alignment analysis of edTPA rubrics and prompts. 



edTPA Rubric 

NGSS Practices 

1 . Asking 
Qs& 
defining 
problems 

2. Developing 
& using 
models 

3. Planning & 
carrying out 
investigations 

4. Analyzing 
& 

interpreting 

data 

5. Using math 
& 

computational 

thinking 

6. Constructing 
explanations & 
descriptions 
solutions 

7. Engaging in 
argumentation 
from evidence* 

8. Obtaining, 
evaluating, & 
communicating 
information 

Task 1: Planning 

1. Planning for Content 
Understandings 






V 

V* 

V 

2. Planning to Support Varied 

Student Needs 









3. Using Knowledge of Students to 
Inform Planning 









4. Identifying and Supporting 
Language Demands 









5. Planning Assessments to Monitor 
and Support Student Learning 









Task 2: 

6. Demonstrating a Positive and 
Engaging Learning Environment 









7. Engaging Students in Learning 




V 


V 


V 

8. Deepening Student Learning While 
Teaching 






V 


V 

9. Subject Specific Pedagogy: 

Science 




V 





10. Analyzing Teaching 









Task 3: Assmt. 

11. Analyzing Student Work 









12. Providing Feedback to Guide 
Learning 









13. Supporting Students’ Use of 
Feedback 









14. Evidence of Language Use to 
Support Content Understandings 






V 



15. Using Assessment to Inform 
Instruction 










*Recommended to be included if there is a reexamination of rubrics and prompts for increased language from NGSS practice: Engaging in 
argumentation from evidence. Modified version retrieved from https://secure.aacte.org 
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